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ABSTRACT 

Limited visibility and reduced situational awareness have been cited as predominant causal factors for both Controlled 
Flight Into Terrain (CFIT) and runway incursion accidents. NASA’s Synthetic Vision Systems (SVS) project is 
developing practical application technologies with the goal of eliminating low visibility conditions as a causal factor to 
civil aircraft accidents while replicating the operational benefits of clear day flight operations, regardless of the actual 
outside visibility condition. A major thrust of the SVS project involves the development/demonstration of affordable, 
certifiable display configurations that provide intuitive out-the-window terrain and obstacle information with advanced 
pathway guidance. A flight test evaluation was conducted in the summer of 2004 by NASA Langley Research Center 
under NASA’s Aviation Safety and Security, Synthetic Vision System - Commercial and Business program. A 
Gulfstream G-V aircraft, modified and operated under NASA contract by the Gulfstream Aerospace Corporation, was 
flown over a 3 -week period at the Reno/Tahoe International Airport and an additional 3 -week period at the NASA 
Wallops Flight Facility to evaluate integrated Synthetic Vision System concepts. Flight testing was conducted to 
evaluate the performance, usability, and acceptance of an integrated synthetic vision concept which included advanced 
Synthetic Vision display concepts for a transport aircraft flight deck, a Runway Incursion Prevention System, an 
Enhanced Vision Systems (EVS), and real-time Database Integrity Monitoring Equipment. This paper focuses on 
comparing qualitative and subjective results between EVS and SVS display concepts. 
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1. INTRODUCTION 

In commercial aviation, over 30% of all fatal accidents worldwide are categorized as Controlled Flight Into Terrain 
(CFIT), where a mechanically sound and normally functioning airplane is inadvertently flown into the ground, water, or 
an obstacle, principally due to the lack of outside visual reference and situational awareness 1 . The Synthetic Vision 
Systems (SVS) project, under NASA’s Aviation Safety and Security Program (AvSSP), is developing technologies with 
practical applications to eliminate low visibility conditions as a causal factor to civil aircraft accidents 2,3 . 

1.1. Synthetic Vision Display Concepts 

A major thrust of the SVS project involves the development and demonstration of affordable, certifiable display 
configurations which provide intuitive out-the-window terrain and obstacle information, including pathway and 
guidance information for precision navigation and obstacle/obstruction avoidance, for Commercial and Business 
aircraft. In addition to forward-fit applications, a path to retrofit this technology into today’s transport aircraft fleet is 
also necessary to achieve the desired safety benefits since 66% of today’s transport aircraft fleet is equipped with only 
electro-mechanical cockpit instrumentation. 

NASA’s SVS concept (Figure 1) provides a real-time, unobscured synthetic view of the world for the pilot. The display 
is generated by visually rendering an on-board terrain database (with additional airport and obstacle database 
information as necessary) using precise position and navigation data obtained through GPS (Global Positioning System) 
data, with augmentation possibly from differential correction sources such as Local Area Augmentation Systems 
(LAAS) and Wide Area Augmentation Systems (WAAS), as well as blending from on-board Inertial Navigation System 
(INS) information. Active imaging sensors, real-time hazard information (e.g., weather and wake vortices), and traffic 
information as provided by Automatic Dependent Surveillance - Broadcast (ADS-B) and Traffic Information Services - 
Broadcast (TIS-B) can additionally enhance this Synthetic Vision Display Concept (SVDC). Although the display 
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representation to the pilot is synthetically derived, object detection and integrity monitoring functions are envisioned to 
ensure sufficient accuracy and reliability for certification. 
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Figure 1: Synthetic Vision System Concept 


1.2. Retrofit Approach 

To measurably reduce the aviation accident rate, SVS display technology must be compatible with and retrofit into as 
many existing aircraft as possible. The retrofit approach that evolved showed the application of SVS into existing head 
down display (HDD) capabilities for aircraft with glass cockpits (cockpits already equipped with raster-capable HDDs) 
and head-up display (HUD) capabilities for the other aircraft. 

Two major NASA flight tests have been conducted for assessment and evaluation of the SVS developments. Both 
flight tests have used the NASA/Langley Research Center (LaRC)-modified Boeing 757-200 jetliner known as the 
Airborne Research Integrated Experiment System (ARIES). The first flight test was flown Sept-Oct 2000 in nighttime 
operations at Dallas-Ft. Worth (FAA Identifier: DFW) 4 . The second flight test was flown Aug-Sept 2001 in simulated 
daylight Instrument Meteorological Conditions (IMC) at Eagle County Regional Airport, CO (FAA Identifier: EGE) 5,6 . 

A cost-effective retrofit path for SVS, particularly for aircraft with electro-mechanical cockpits, may be possible by 
generation of a synthetic vision image as the raster input source to a stroke-on-raster HUD. This display concept is 
analogous in many respects to the Enhanced Vision System (EVS) certified on the Gulfstream V, except that the raster 
image is synthetically-derived rather than being a direct imaging sensor output. Unlike EVS displays, the SVS HUD 
concept does not generate any raster HUD image above the terrain (i.e., it shows a “clear sky”) as opposed to a sensor 
image of the sky. 

In both flight trials, the SVS-HUD concept was, for all intents, just a monochromatic green representation of the full- 
color, head-down display SVS concept, using an RS-343 video format. SVS HUD concept evaluations included two 
variations in the method of terrain texturing (photo-realistic and generic) that were also evaluated in the head-down 
displays. 5,6 In order to create a 3-dimensional perspective terrain appearance, graphical lighting is used to create 
shading when the terrain database is generated. This shading of the terrain was not optimized for rendering on a 
monochrome HUD. The feasibility of the concept of retrofitting SVS display technology with HUDs was verified, 
particularly for nighttime operations. Pilots reported greater situation awareness while operating with the SVS-HUD 
concepts compared to the conventional displays. Although promising results were obtained, two significant deficiencies 
were found in daylight HUD usage: illegible display renditions under some direct sunlight conditions and some reported 
terrain depiction illusions particularly from the terrain shading. For both HDD and HUD applications, no significant 
performance effects were found between the two terrain texturing techniques, although most of the pilots preferred the 
photo-realistic texturing technique. 



As a result of these comments, a hybrid terrain texturing concept was developed. Pilots commented that they liked the 
photo-realistic terrain as it provided subtle terrain patterns used naturally during VFR flying and significant cultural 
feature information. They also commented that they liked the absolute altitude cue of the generic terrain as it provided 
better awareness of the surrounding terrain height. Therefore, a hybrid texturing technique was developed which 
combines the generic terrain altitude cue with the photo-realistic terrain. The resulting database has been highly rated 
by pilots and is now use as the terrain texturing concept for SVS displays. The SVS database for all HUD applications 
is now tailored to eliminate the aforementioned terrain illusions. 

1.3. Current Flight Test Study 

A flight test, known as Gulfstream-V Synthetic vision Integrated Technology Evaluation (GVSITE), was conducted at 
Reno, NV (FAA identifier RNO) and Wallops Island, VA (FAA identifier WAL) during the summer of 2004 on board 
an experimental Gulfstream aircraft. The purpose of the flight test was to evaluate the integration of several SVS 
enabling technologies. These enabling SVS technologies include: SVDC consisting of computer generated terrain 
databases with advanced guidance systems, Runway Incursion Prevention System (RIPS), terrain Database Integrity 
Monitoring Equipment (DIME) and weather radar runway detection systems known as SV-Sensors. Up to this point, 
the various parts that make up the complete SVS concept were tested as independent components. 

This paper focuses on the test results for the SVDC terrain and advanced guidance concepts, emphasizing a comparison 
between the SVS and EVS concepts. The integrated results as well as the other enabling technologies are discussed in 
References 7-9. 

1.4. Experiment Objectives 

The primary GVSITE flight test objective was to evaluate the utility and acceptance of an integrated Synthetic Vision 
System intended for commercial and business aircraft in a terrain-challenged operational environment. Although 
specific experiment objectives for the individual technology areas of SVDC, RIPS, DIME and SV-Sensors were also 
evaluated, only those associated with SVDC will be discussed in this paper. 

The SVDC’s provide the human-machine interface to the synthetic vision system. Display elements include, for 
example, perspective terrain, flight path guidance, and traffic information both in the air and on the surface presented on 
multiple display surfaces (HUD, Primary Flight Display (PFD), and Navigation Display (ND)). 

The primary SVDC flight test objective was to investigate the operational utility and acceptability of enhanced terrain 
awareness of SVS display concepts to support Required Navigation Performance (RNP)-like approach procedures in a 
terrain-challenged operational environment. This test was designed to assess flight path control performance, pilot 
workload, and situation awareness during manually-flown landing approach, with and without SVS display concepts, 
and thus, determine the effect on that performance of the presence of SVS components. 

Of particular interest was the evaluation of several improvements made to the SVDC’s based on previous ground 
simulation and flight test activities: 

• SVS advanced guidance concepts (flight path error indicators, pursuit guidance and 3D perspective flight path 
tunnel) have been implemented in the stroke symbology of the HUD which greatly improve the readability of 
the guidance symbology in both day and night conditions. 

• A dynamic tunnel concept was developed using ground simulation which dynamically changes in appearance 
based on the flight path error. 11,12 

• The terrain databases were implemented not only on the HUD and PFD but also on the ND using Warning And 
Caution Overlays driven from Terrain Awareness and Warning System inputs. 

• All SVS terrain databases used the hybrid texture technique which combines the advantages of photo-realistic 
and generic textured databases. 

• Display Symbology transitions were developed based on phase of flight. 

• RIPS and DIME alerts were integrated in the SVDC’s. 



2. METHOLOGY 


2.1. Gulfstream-V Experimental Aircraft 

This flight test was conducted on board an experimental Gulfstream-V (G-V) S/N 501 aircraft. The evaluation pilot 
flew in the left seat and the safety pilot was in the right seat. Three experimental displays were installed on the G-V for 
the evaluation pilot flying the aircraft (see Figure 2): a monochrome head-up display (HUD) and two color, Size D (8 
inches x 8 inches) head-down displays. 

The HUD was an experimental Flight Dynamics HGS-4000 system. The HGS HUD computer was modified to display 
additional advanced guidance symbology in stroke format. The HUD also had a raster channel input which was used to 
display computer-generated terrain or forward looking infrared (FLIR) imagery. The FLIR camera was a standard G-V 
Kollsman FLIR camera. The cryogenically-cooled FLIR camera operates in the 1.3 to 4.9 micron wavelength using a 
sensor with approximately 320 H x 240 V pixel resolution. To simulate IMC type conditions for some of the evaluation 
data runs, a vision restriction device (VRD) was placed behind the HUD to block the forward vision through the HUD. 
Also, HUD combiner sunvisors were fabricated from four different shades of neutral density (gray) cast acrylic and 
installed as necessary. The sunvisors were attached to the HUD combiner by Velcro to allow easy installation and 
removal. The sunvisors provide improved raster readability by increasing the contrast ratio through ambient light 
attenuation. 



Figure 2: G-V In-Flight and Cockpit View Showing the HUD, SVS PFD, and SVS ND. 


2.2. Test Subjects 

Ten evaluation pilots (EPs), representing the airlines, a major transport aircraft manufacturer, the FAA and the Joint 
Aviation Authority, participated in the experiment. One hundred and forty-five flight test runs were conducted to 
evaluate the NASA SVS concepts at WAL (8 pilots) and RNO (7 pilots) airports. Five of the ten EPs flew at both test 
locations. All participants were experienced in heavy twin jet operations and HUD-experienced. All of the test subjects 
were trained in the Integrated Flight Deck (IFD) 757 flight simulator at NASA Langley. Though the actual flight trials 
were conducted onboard a Gulfstream aircraft, the display concepts and scenarios presented in the IFD were 
representative of the scenarios conducted during the actual flight trials. In addition, the subjects were given a 1 hour 
briefing just prior to the evaluation flights to explain the purpose of the flight test, explain the expected task for the 
flights and refresh the pilots on the display concept material. Subject pilots were then given 4 training runs to 
familiarize themselves with the aircraft and experiment scenarios. The in-flight training was immediately followed by 
12 data collection trials. Each flight lasted approximately 3 hours. 

3. EXPERIMENTAL TEST 


3.1. Evaluation Tasks 

There were 2 primary flight tasks performed by the subject pilots: 1) manually fly an approach procedure with auto- 
throttles and 2) aircraft surface operations. This paper focuses solely on the approach task. 



Pilots were instructed to fly the approach path by following the guidance 10 presented to them. Pilots were to land the 
airplane unless they determined that a go-around should be executed. After touchdown, the displays transitioned to the 
RIPS display concepts for landing roll-out and other surface operations. 

The primary approach flown by the evaluation pilots was the Sparks East visual arrival to RNO Runway 16R. 
However, due to winds or traffic, several alternate approaches were also flown such as the Sparks North arrival to RNO 
16R, the South Hills East arrival to RNO Runway 34L and the South Hills South to RNO Runway 34L. What makes 
these visual approaches challenging is that they are designed to avoid noise sensitive areas and obstacles around the 
Reno area, resulting in curving, descending approaches. These same geometries were duplicated to create flight plans 
to the runways at WAL. 

3.2. Display Conditions 

Four different display concepts were evaluated (see Figure 3): 

1) A baseline concept (Figure 4) which represents display technology analogous to current aircraft (“Baseline”); 

2) The baseline concept with the addition of FFIR on the HUD raster channel (“Baseline & FFIR”); 

3) SVS-Enhanced Head-Down Displays (“SVS-HDD”, shown in Figure 5); and, 

4) SVS-Enhanced HUD and Head-Down Displays (“SVS-HUD/-HDD”). 
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Figure 3: The Four Display Concepts Evaluated for GVSITE. 

All display concepts had a velocity vector, an integrated flight guidance cue and path deviation indicators. The SVS 
display concepts used a tadpole symbol as its integrated flight guidance cue. The tail on the tadpole provides track 
change information by rotating left or right in the direction of the desired lateral path. For the baseline concepts, the 





integrated flight guidance cue was identical to the SVS tadpole except the tadpole tail was removed. Thus, each concept 
used the identical guidance control law but no track change information on the baseline integrated flight guidance cue 
was presented to the pilot. 

The path deviation indicators showed “raw data” information (vertical and lateral path error) as well as glideslope and 
localizer deviation (when available) for all the display conditions using error data scaled in “dots”. On the SVS display, 
in addition to the path deviation indicators, a dynamic tunnel concept was displayed which is a 3 dimensional 
representation of the flight path on the PFD and HUD. 11, 12 

The SVS displays also had an auto symbology transition feature that would present or remove various display 
symbologies depending on phase of flight. For example, the dynamic tunnel was removed at 200 feet above field level. 



Figure 4: Baseline Airborne Head-Down Displays. 



Hybrid 

Textured 

Terrain 


Vertical 
Situation 
Display (VSD) 


Figure 5: SVS Airborne Head-Down Displays. 


3.3. Measures 

After each run, the EPs completed a run questionnaire consisting of the Air Force Flight Technical Center (AFFTC) 
Revised Workload Estimation Scale 13 (Figure 6) and eight Likert- type (7-point) questions specific to the display 
concept evaluation (Figure 7). After each flight, the EPs completed two separate Situation Awareness - Subjective 
Workload Dominance (SA-SWORD) 14 and Subjective Workload Dominance (SWORD) 15 tests (to evaluate situation 
awareness and workload, respectively): one for display concept (Baseline, Baseline & FLIR, SVS-HDD, SVS-HUD/- 
HDD) comparisons during approach and another for display concept comparisons during surface operations. The EPs 
also participated in a semi- structured interview after the flight test to elicit comments on pilot preferences of the display 
concepts and subjective assessments of the symbology presentation and transitions. 






Workload Estimate 

1 

Nothing To Do; No System Demands 

2 

Light Activity; Minimum Demands 

3 

Moderate Activity - Easily Managed; Considerable Spare Time 

4 

Busy - Challenging but Manageable; Adequate Time Available 

5 

Very Busy - Demanding To Manage; Adequate Time Available 

6 

Extremely Busy- Very Difficult; Non-Essential Tasks Postponed 

7 

Overloaded - System Unmanageable; Essential Tasks Undone; Unsafe 


Figure 6: AFFTC Workload Scale. 


Q1. As I performed the Evaluation Task, The Effects of Winds and Turbulence were . 

Q2. As I performed the Evaluation Task, The Level of Display Clutter on the HUD was . 

Q3. As I performed the Evaluation Task, The Level of Display Clutter on the HDDs was . 

Q4. As I performed the Evaluation Task, My Ability to Determine Aircraft Position 

With Respect to Terrain was . 

Q5. My Confidence in the Synthetic Vision Scene (Terrain, Obstacles, Airport) was . 

Q6. As I performed the Flying Task, My Level of Flight Path Control and Performance was _ 
Q 7. For Surface Operations, My Level of Situation Awareness Using the 
Surface Display Concept was . 

Q8. The Acceptability of the Lead Time Provided by the Runway Incursion Alert was . 


Scale Used for Eight Post-Run Questions 
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Figure 7: Post-Run Questions for GVSITE Flight Experiment. 

4. RESULTS 


4.1. Quantitative Flight Path Tracking Performance 

Root-mean-square (RMS) calculations of lateral and vertical path error were used as the measures for flight path 
performance. The calculations began when the pilot first entered the tunnel boundaries and ended at 500 feet above 
field level. Simple analysis of variances (ANOVAs) and Student-Newman-Keuls (SNK) post-hoc tests with alpha set at 
0.05 were performed. 

4.1.1. Lateral and Vertical Path Error Data 

For this analysis, the independent variables were display type (Baseline, Baseline & FLIR, SVS-HDD, SVS-HUD/- 
HDD), path type, and pilot. The dependent measures were RMS lateral path error and RMS vertical path error. 
Separate ANOVAs were performed on the dependent (path error) variables. 

Display type, path type, pilot, and the second order interactions between the main factors were not significant (p>.05) 
for either measure. 

4.1.2. RNP Lateral Navigation Analyses 

In addition, the flight path tracking performance was used to determine what level of RNP 17 they could achieve while 
hand-flying the approach. In the following analysis, the only error source being considered in this RNP analysis is the 
Flight Technical Error (FTE) contribution to path steering error, where FTE is the accuracy with which the aircraft is 
controlled as measured by the indicated aircraft position with respect to the indicated desired position. 

Lateral path FTE histograms were generated on the nominal approach runs for the four display concepts (Baseline, 
Baseline & FLIR, SVS-HDD, SVS-HUD/-HDD). Since the initial starting position for each run was outside the tunnel, 


the histogram analyses were not initiated until the pilot had entered the tunnel boundary for the first time. The analysis 
ended at 500 feet above field level. 

With these analysis assumptions, the Baseline & FLIR and both SVS concepts were shown to yield a lateral FTE (RNP) 
navigational accuracy of 0.04 nm at least 95% of the time; while, the Baseline concept yielded a lateral FTE (RNP) 
navigational accuracy of 0.1 nm at least 95% of the time. 

4.2. Subjective Results 

4.2.1. Post Run Questionnaire 

AFFTC workload estimate rating (Figure 6) and eight post-run questions (Figure 7) were asked of each EP while 
performing airborne and surface operations with the display concepts. An ANOVA was performed on the mean ratings 
with pilot, display concept, and Path Type (when appropriate) as the independent variables. 

4.2.1. 1. Head Down Display Clutter 

Statistically significant differences were not found in the answers to the questions pertaining to flight path performance 
(Question 5), terrain awareness (Question 4), or Head-Up Display Clutter (Question 2). 

Only post-run Question 3 (F(3,54) = 6.564, p < .01) had significant differences among the display concepts evaluated. 
For post-run Question 3, each subject was asked to assess the level of display clutter on the head-down displays. Post 
hoc tests (using SNK with a= .05) showed two unique subsets: 1) SVS-HDD (3. 0/7.0) and 2) Baseline (1. 5/7.0), 
Baseline & FLIR (1. 6/7.0), and S V S-HUD/-HDD (1. 6/7.0). 

4.2.1.2. AFFTC Workload Estimate Ratings 

There were no statistically significant differences (p > .05) for the Air Force Revised Workload Estimation Scale 
amongst the display concepts. (This workload estimate was provided immediately following each run.) Pilots rated 
their workload on average from “light” (SVS-HUD/-HDD) to “moderate activity” (Baseline). 

4.2.2. Workload 

Post-flight, each EP completed a paired-comparison SWORD 15 for approach operations to allow a statistical analysis of 
the pilot’s subjective assessment of mental workload for each of the display concepts. For this exercise, mental 
workload was defined as: The amount of cognitive resources available to perform a task and the difficulty of that task. 
The responses were averaged and the overall rank order from least to greatest workload for airborne operations was: 
SVS-HUD/-HDD, SVS-HDD , Baseline & FLIR, and Baseline. 

An ANOVA was performed on the mean rankings with display concept and pilot as the independent variables. Display 
concept was significant (F(3,33) = 8.470, p < .05) for this measure. Post hoc tests (using SNK, with a = .05) showed 
that the Baseline concept had significantly higher mental workload ratings than all other display concepts tested. 

4.2.3. Situation Awareness 

Post-flight, each EP completed a paired-comparison SA-SWORD 14 for approach operations to allow a statistical 
analysis of the pilot’s subjective assessment of the situation awareness (SA) for each of the display concepts. For this 
exercise, SA was defined as: The pilot’s awareness and understanding of all factors that will contribute to the safe 
flying of their aircraft under normal and non-normal operations. The responses were averaged and the overall rank 
order from greatest to least SA was: SVS-HUD/-HDD, SVS-HDD, Baseline & FLIR, and Baseline. 

An ANOVA was performed on the mean rankings with display concept and pilot as the independent variables. Display 
concept was significant (F(3,27) = 8.188, p < .05): for this measure. Post-hoc tests (using SNK, with a = .05) revealed 
two unique subsets for display concept comparisons of situation awareness during approach: (1) SVS-HUD/-HDD 
(highest SA) and (2) SVS-HDD, Baseline & FLIR, and Baseline (lowest SA). 



4.2.4. Semi Structured Interview 


Post-flight, each EP rank-ordered the display concepts in terms of: (a) flight path performance and awareness; and, (b) 
their preference if they were doing an IMC approach. A Friedman test (p < .05) evinced a significant ranking for both 
questions in the order of: (1) SVS-HUD/-HDD (highest rating); (2) SVS-HDD; (3) Baseline & FLIR; and (4) Baseline 
(lowest rating). 


5. DISCUSSION 

5.1. Quantitative Flight Path Performance Data 

No significant differences were found among the four display concepts (Baseline, Baseline & FFIR, SVS-HUD/-HDD, 
SVS-HDD) for the measures of flight path performance using RMS path error. However, the baseline configuration 
was found to be significantly worse in lateral RNP than the other three display concepts. In operational terms, the 
differences are small (0.04 nm FTE vs. 0.1 nm FTE) when compared to current aircraft equipage that shows FTE on the 
order of 0.25 nm. 16 

The primary differences between these concepts, with respect to flight path performance, was the HUD or HDD 
background (none, EVS, or SVS), the presence/absence of the tunnel, and the presence/absence of track change 
information (tadpole or dual cue guidance symbol). 

The lack of operationally-significant differences in flight path performance results between the display concepts is not 
surprising. These findings, supported by past research, 11,12,16 confirm that: 

• The presence or absence of terrain (synthetic or enhanced) is not a dominant factor in flight path performance 
when compared to the influence of flight guidance symbology. Each display concept utilized the same pursuit 
guidance control laws and symbology (i.e., the flight path marker, integrated guidance symbol, and path 
deviation indicators which commanded the pilot where to fly). By design, this experiment did not test the 
ability of pilots to navigate solely with respect to the terrain (background) information. 

• Pilot comments indicated that the tadpole did provide some additional information during the approach task but 
it was not a significant factor in navigation performance. 

• The addition of the tunnel concepts in the advanced display formats was not significant in this quantitative path 
performance data analysis, but did influence the subjective workload and SA measures. In the subjective 
ratings and rankings, the presence of tunneFpathway information provided significant (situation) path 
awareness and reduced pilot workload. 

The FTE results should be qualified in that they did not neatly capture the influence that the different guidance and 
tunnel symbologies may have produced with off-path starting conditions, because it was not possible to precisely 
control the run- start conditions and ATC-directed maneuvers in the dynamic air traffic/flight test environment; thus, the 
FTE results were normalized by using the tunnel intercept condition (whether the tunnel was explicitly shown or not) to 
begin the FTE “scoring.” 

The lack of quantitative flight path performance differences between the configurations was also confirmed by the post- 
run pilot ratings. Post-run Question 5, related to flight path performance, did not yield any statistically significant 
differences between the configurations. 

5.2. Subjective Rating Data 

The only post-run question that was of statistical significance showed that clutter increases on the head-down display 
when SVS is added to it and when this display is used as the primary flight reference. Although the ratings indicate 
more clutter on the SVS head-down display, the pilots still rated the SVS-HDD concept as providing substantial 
improvements in SA, without a concomitant increase in workload, compared to the Baseline concepts. 

No statistically significant differences were found in the answers to the post-run questions pertaining to Head-Up 
Display Clutter (Question 2), although the baseline (no raster on the HUD) was rated as having the least clutter (mean 



rating of 1.6) compared to the HUD with FLIR (mean rating: 2.7) and HUD with SVS (mean rating: 2.8). These results 
suggest the intuitively obvious - adding raster information to the HUD produces display clutter. In this flight test, FLIR 
and SVS are essentially equivalent in the amount of clutter added. The flight test was conducted almost exclusively in 
VMC (due to the time of the year for testing) so the influence of weather on FLIR performance and its effect on the 
HUD clutter was not evaluated. 

No statistically significant differences were found in the answers to the post-run questions pertaining to terrain 
awareness (Question 4). This result is moderately surprising given the terrain-challenged environment at RNO, 
however, the side windows were not obstructed and the EP’s were instructed that they could include that terrain 
awareness in their responses. Very significant terrain awareness differences have been found between similar display 
configurations in other flight tests. 5 

5.3. SA, Workload, and Preference for SVS versus EVS 

In Figure 8, a summary of the significant results from the post- flight subjective ratings is shown comparing the display 
concepts for workload, situation awareness, and pilot preference. The numbers indicate the post-hoc subsets as 
determined by the ANOVA analyses. 

This summary shows that the SVS-HUD/-HDD concept provides higher situational awareness and lower mental 
workload than the other three concepts. 2,4,5,12 Pilot comments indicated that the SVS-HUD/-HDD display concept 
provided the pilots’ with an intuitive mental model of the aircraft situation with respect to terrain, other traffic, and the 
intended flight path. Intuitive display of this information has been a primary goal for NASA SVS display development. 
In addition, pilots rated the SVS-HUD/-HDD concept as having the same display clutter level as the Baseline and 
Baseline & FLIR concepts. (Though the SVS-HUD/-HDD concept and the SVS-HDD concept used the same head 
down displays, the SVS-HDD was rated as having significantly higher clutter than the other three concepts. This 
highlights that the pilots are flying mostly using the HUD for the SVS-HUD/-HDD concept and only referencing the 
head-down displays.) 

These results suggest that adding FLIR to a HUD did not provide higher situational awareness than either the SVS head- 
down only (SVS-HDD) or SVS on all displays configuration (SVS-HUD/-HDD). 


SWORD 

SA-SWORD 

Pilot Preference (Post-Flight) 

1) SVS-HUD/-HDD 

1) SVS-HUD/-HDD 

1) SVS-HUD/-HDD 

SVS-HDD 

2) SVS-HDD 

2) SVS-HDD 

Baseline & FLIR 

3) Baseline & FLIR 

3) Baseline & FLIR 

2) Baseline 

Baseline 

4) Baseline 


Figure 8: Summary of SVDC Post-Hoc Test Subsets. 


Several factors contributed to these findings, as noted in the following: 

• The EPs preferred the HUD concepts in general (with SVS or EVS), as they provide a superior display to 
obtain the required flight information and yet, remain head-out to see traffic, weather, and terrain (when 
visible). In day VMC, the HUD sunvisors helped daylight readability of the HUD (SVS or FLIR) raster. 
However, readability was not always perfect so, in the case of SVS, the pilots were able to go head-down, and 
obtain terrain and pathway information. 

• While flight in IMC was not conducted often, the EPs were familiar with FLIR’s inability to penetrate all 
weather conditions. EPs noted how clouds were visible and could distort their impression of the surrounding 
terrain - unlike SVS. Thus, their ratings were sometimes down-graded because the FLIR could not provide 
terrain information, accurately and reliably in all weather, unlike SVS. Conversely, the SVS display can be 
tailored to ensure and promote terrain awareness; the SVS terrain design on the HUD and HDD for GVSITE 
was an improvement over previous NASA SVS implementations. 5 

• The SVS concepts, unlike the FLIR concepts, included pathway or tunnel information. While some 
disagreement between EPs was evident in the tunnel design and its implementation, the value of the tunnel for 


path awareness was generally agreed upon by all for the descending, turning curved approach application. 
Once stabilized on final, the EPs preferred that the tunnel be removed. The tunnel didn’t offer value 
(performance benefit) to offset the display clutter increase. For this flight test, the tunnel decluttered at 200 ft 
AFL. The automatic declutter was “a nice feature”, but the altitude was too low. 

• Runway object/obstacle alerting for the SVS concepts was a significant benefit. The intruding test vehicles 
were equipped with ADS-B and the SVS concepts used symbolic representations of this sensor data to indicate 
on the HDD or HUD SVS displays precisely where these vehicles were located. Incursions, if they occurred, 
also triggered RIPS audio and visual alerts. These components greatly increased SA. For FFIR evaluations of 
runway incursions, the EP, as the pilot- flying, was solely responsible for traffic awareness and obstacle/object 
detection. Numerous instances occurred where the incurring traffic was not seen by the EP using the FFIR. In 
addition, while FFIR can provide an element of traffic detection above and beyond unaided flight, FFIR is still 
limited by its inherent resolution, the size and thermal characteristics of the obstacle/object against the 
background, and the inherent display resolution of the HUD. 

5.4. SVS or EVS? 

While these data suggest a clear preference for SVS compared to the FFIR (EVS) concepts, pilot comment data and an 
understanding of the experimental design shows that the true answer isn’t SVS or EVS, but rather, SVS and EVS. 

Experimental evidence has shown that a “perfect” SVS, which includes decision aides for alerting for runway incursion 
and database integrity, is superior to SVS concepts without alerting or EVS concepts. 18 In this flight test, decision 
aiding for DIME and RIPS was included with the SVS concepts to near perfection and proved to be superior to FFIR 
(EVS) concepts. While RIPS and DIME elements are developing technology to support the “perfect” SVS application, 
there may always be “imperfections” in the technology (e.g., non-squawking traffic for RIPS, finite precision in sensors 
and database accuracy for DIME), at least for the near-future. 

SVS, by being weather- independent and full field-of-regard, holds many advantages over forward looking sensor 
systems for terrain, path, and obstacle awareness in the many flight phases (particularly during the approach). Several 
pilots suggested this superiority is maintained (without reservation) on the approach until the “final approach fix” or a 
“stabilized on approach” point. 

Beyond this point on the approach, the need for EVS becomes more prevalent in the absence of a “perfect” SVS. 
Several EPs noted this reservation and pointed out that a FLIR image provided them confidence in the SVS imagery and 
that this was an additional integrity sensor as a complement to SVS technology. 

Research has begun to develop integrated and fused EVS and SVS technologies to create “the best of both worlds.” 

6. CONCLUSIONS 

The flight test marked the first time NASA’s technologies have been integrated as a complete system incorporating 
synthetic terrain primary flight and navigation displays, advanced weather radar object detection, synthetic vision 
database integrity monitoring, refined dynamic tunnel and guidance concepts, surface map displays, and the runway 
incursion prevention system (RIPS). The results showed the efficacy of the NASA Synthetic Vision System to 
significantly enhance pilot situation awareness (without increasing mental workload) for runway traffic and terrain, and 
substantially better pilot acceptability and trust due to integrated integrity monitors and enhanced vision sensors. 
Although many of the evaluation pilots had never flown a Gulfstream V aircraft, they were able to fly within .04 nmi 
lateral RNP with the SVS displays. In addition, pilots reported a lower workload and increased situational awareness 
compared to the Baseline display concepts. The results are consistent with results from previous experiments and flight 
test trials conducted by NASA. 

Future SVS research will focus on (1) enhancement of the dynamic tunnel concept to provide 4-D required time of 
arrival and required navigation performance, (2) crew coordination human factors research using SVS and fused 
SVS/EVS technologies, (3) exocentric dynamic 3-D SVS navigation displays for approach and missed approach 
rehearsal, (4) military applications of synthetic vision, (5) advanced display media, and, (6) the integration of SVS with 
other emerging NASA cockpit information displays. 
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